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(54) INTERACTIVE DEVICE 

(57) An interactive apparatus 1 which is able to de- 
cide on an action pattern in accordance with health con- 
ditions of a user without a necessity of putting a biomet- 
ric sensor on a human body is provided. The interactive 
apparatus 1 comprises: detection means 50b for detect- 
ing a health condition of a user; deciding means 50c for 
deciding on an action pattern in accordance with the 
health condition of the user, execution instructing 
means 50g for instructing execution of the action pat- 
tern; offering means 50e for making an offer of the action 
pattern to the user with a speech before instructing ex- 
ecution of the action pattern; and determination means 
50f for determining whether an answer of the user to the 
offered action pattern is an answer to accept the offered 
action pattern or not The execution instructing means 
50g instructs execution of the offered action pattern 
wh en the answer of the user is determined to be the an- 
swer to accept the offered action pattern. 



FI6.5 



ST1 




Offer action pattern to umt 



-ST4 



$75 




STB 



/ST7 



Store action pattam and tot that 
oflbr h not accepted at Matey 



Q. 
til 



Printed by Jouve, 75001 PARIS (FR) 



1 



EP 1 542 101 A1 



2 



Description 

TECHNICAL FIELD 

[0001] The present invention relates to an interactive 
apparatus which can have a conversation with a user. 

BACKGROUND ART 

[0002] An audio apparatus which monitors for behav- 
ior information for reproducing an audio signal of pref- 
erence of a sequential habitant at a level adjusted in ac- 
cordance with the current situation and the physical con- 
dition of the user (see, for example, Japanese Laid- 
Open Publication No. 11-221196). The audio apparatus 
detects the situation of the habitant by using a sensor 
provided in a room. The audio apparatus monitors iden- 
tification information and behavior information from a 
portable transceiver (including a btometric sensor) worn 
by the habitant, and adjusts the audio signal of the pref- 
erence of the sequential habitant to a level in accord- 
ance with the current situation and the physical condi- 
tion of the habitant for reproduction. 
[0003] However, in the conventional art as described 
in Japanese L aid-Open Publication No. 11-221196, the 
habitant has to wear a portable transceiver for acquisi- 
tion of biometric information and the like. Wearing the 
sensor is cumbersome for the habitant, and thus, this 
method is inconvenient There is also a problem that the 
habitant is monitored all the time by the sensor provided 
in the room, which may cause uncomfortable feeling of 
the habitant 

[0004] The object of the present invention is to provide 
an interactive apparatus which is able to decide on an 
action pattern in accordance with the health conditions 
of the user without a necessity of putting a btometric sen- 
sor on a human body. 

DISCLOSURE OF THE INVENTION 

[0005] An interactive apparatus according to the 
present invention comprises: detection means for de- 
tecting a health condition of a user; deciding means for 
deciding on an action pattern in accordance with the 
health condition of the user detected by the detection 
moans; execution instructing means for instructing ex- 
ecution of the action pattern decided by the deciding 
means; offering means for making an offer of the action 
pattern to the user with a speech before instructing ex- 
ecution of the action pattern decided by the deciding 
means; and determination means for determining 
whether an answer of the user to the offered action pat- 
tern is an answer to accept the offered action pattern or 
not, in which the execution instructing means instructs 
execution of the offered action pattern when the answer 
of the user is determined to be the answer to accept the 
offered action pattern, thereby achieving the above-de- 
scribed object. 



[0006] The detection means may detect the health 
condition of the user based on utterance of the user 
[0007] The detection means may detect the health 
condition of the user based on keywords uttered by the 
5 user. 

[0008] Offer necessity determi nation means for deter- 
mining whether it is required to make an offer of the ac- 
tion pattern to the user before instructing execution of 
the action pattern decided by the deciding means may 

10 be further included, and the offering means may make 
an offer of the action pattern to the user with a speech 
when it is determined that making an offer of the action 
pattern to the user is required before instructing execu- 
tion of the action pattern. 

15 [0009] The offer necessity determination means may 
determine necessity of making an offer in accordance 
with a value of a flag indicating a necessity of making 
an offer which is previously allocated to the action pat- 
tern. 

20 [0010] The offer necessity determination means may 
determine necessity of making an offer based on time 
distribution of the number of times the action pattern is 
performed. 

[001 1 ] The deciding means may decide one of a plu- 

25 ralrty of action patterns to which priorities are respec- 
tively allocated as an action pattern in accordance with 
the health condition of the user, and may change the 
priority allocated to the action pattern in accordance with 
whether or not the action pattern is accepted by the user. 

30 [0012] Storage means for storing the action pattern in 
accordance with the health condition of the usermay be 
further included, and the deciding means may decide 
on the action pattern by using the action pattern stored 
in the storage means. 

3s [0013] The action pattern offered by the offering 
means to the user may include selecting contents to be 
reproduced by a reproducing device. 
[0014] The contents may include audio data, video 
data, and lighting control data, and the reproducing de- 

-*o vice may change at least one of light intensity and color 
of light of a lighting apparatus based on the lighting con- 
trol data. 

[0015] The interactive device may have at least one 
of an agent function and a traveling function. 
*s [0016] The health condition of the usermay represent 
at least one of feelings of the user and a physical con- 
dition of the user. 

[0017] An interactive apparatus according to the 
present invention comprises: a voice input section for 
so converting a voice produced by the user into a voice sig- 
nal, a voice recognition section for recognizing words 
uttered by the user based on the voice signal output from 
the voice input section; a conversation database in 
which words expected to be uttered by the user are pre- 
ss viously registered, and which stores correspondences 
between the registered words and the hearth condition 
of the user; detection means for detecting the hearth 
condition of the user by checking the words recognized 



2 



3 



EP 1 542 101 A1 



4 



by the voice recognition section against the words reg- 
istered in the conversation database, and deciding on 
the health condition of the user in accordance with the 
checking result; deciding means for deciding on an ac- 
tion pattern in accordance with the health condition of 
the user detected by the detection means based on an 
action pattern table storing correspondences between 
the health condition of the user and action patterns of 
the interactive apparatus; execution instructing means 
for instructing execution of the action pattern decided by 
the deciding means; offering means for synthesizing an 
offering sentence based on an output result of the de- 
tection means and an output result of the deciding 
means and making an offer of the action pattern to the 
user with a speech before instructing execution of the 
action pattern decided by the deciding means: and de- 
termination means for determining whether an answer 
of the user to the offered action pattern is an answer to 
accept the offered action pattern or not, in which the ex- 
ecution instructing means instructs execution of the of- 
fered action pattern when the answer of the user is de- 
termined to be the answer to accept the offered action 
pattern, thereby achieving the above-described object. 
[001 8] Means for receiving an action pattern which is 
counter-offered by the user with respect to the offered 
action pattern, means for the interactive apparatus to 
determine whether the counter-offered action pattern is 
executable or not, and means for updating the corre- 
spondences between the health condition of the user 
and the action patterns of the interactive apparatus 
which are stored in the action pattern table when the 
interactive apparatus determines that the counter-of- 
fered action pattern is executable may be further includ- 
ed. 

BRIEF DESCRIPTION OF THE DRAWINGS 
[0019] 

Figure 1 is a diagram showing an appearance of a 
robot 1 as an example of an interactive apparatus 
according to the present invention. 

Figure 2 is a diagram showing an exemplary inter- 
nal structure of the robot 1 . 

Figure 3 is a diagram showing exemplary relation- 
ships between keywords to be generated by a user 
which are stored in a conversation database 140 
and the health conditions of the user. 

Figure 4 is a diagram showing exemplary relation- 
ships between the health conditions of the user 
which are stored in an information database 160 
and an action pattern of the robot 1. 

Figure 5 is a flow chart showing an exemplary pro- 
cedure for the robot 1 to detect the health condition 



of the user and indicate execution of an action pat- 
tern which matches the hearth condition of the user. 

Figure 6 is a diagram showing an exemplary struc- 
5 ture of a reproducing apparatus 21 00 which allows 
synchronized reproduction of audio data and/or vid- 
eo data, and lighting control data. 

Figure 7 is a diagram showing an exemplary inter- 
na nal structure of a voice recognition section 40. 

Figure 8a is a diagram showing an exemplary inter- 
nal structure of a processing section 50 shown in 
Figure 2. 

15 

Figure 8b is a diagram showing another exemplary 
internal structure of the processing section 50 
shown in Figure 2. 

20 Figure 8c is a diagram showing another exemplary 
internal structure of the processing section 50 
shown in Figure 2. 

Figure 9 is a diagram for illustrating how offering 
25 means 50e create offering sentences. 

Figure 10 is a diagram showing an exemplary inter- 
nal structure of offer necessity determination means 
50d. 

30 

Figure 1 1 is a diagram showing an exemplary struc- 
ture of an action offer necessity table 162. 

BEST MODE FOR CARRYING OUT THE INVENTION 

35 

[0020] Hereinafter, the embodiments of the present 
invention will be described with reference to the draw- 
ings. 

[0021] As used herein, a "health condition of a user" 
40 refers to at least one of the feeling or a physical condition 
of a user A "user" refers to an owner of the interactive 
apparatus. 

[0022] Figure 1 shows an appearance of a robot 1 as 
an example of an interactive apparatus according to the 

45 present invention. The robot 1 is formed such that it can 
have conversation with a user. 
[0023] The robot 1 shown in Figure 1 includes: a cam- 
era 10 which corresponds to an "eye"; a speaker 110 
and an antenna 62 which correspond to a "mouth"; a 

so microphone 30 and an antenna 62 which correspond to 
an "ear"; and movable sections 180 which correspond 
to a "neck" and an "arm". 

[0024] The robot 1 may be an autonomous traveling 
robot (a mobile robot) having traveling sections 160 
55 which allows it to travel by itself, or may be of a type 
which cannot be moved by itself. 
[0025] Any mechanism may be adopted as a mecha- 
nism for allowing the robot 1 to travel. For example, the 



5 



EP 1 542 101 A1 



6 



robot 1 may be formed so as to move forward or back- 
ward by controlling rotations of rollers provided on 
hands and feet. Alternatively, the robot 1 may be a mo- 
bile robot using tires or legs. The robot 1 may be a hu- 
man-shaped robot which imitates an animal walking up- 5 
right with two legs such as human, or may be a pet robot 
which imitates an animal walking with four legs. 
[0026] The interactive robot has been i llustrated as an 
example of interactive apparatuses. However, the inter- 
active apparatuses are not limited to this. The interactive 10 
apparatuses may be any apparatus formed such that it 
can have a conversation with users. The interactive ap- 
paratuses may be, for example, interactive toys, inter- 
active portable devices (including mobile phones), or in- 
teractive agents. *5 
[0027] It is preferable that the interactive agents have 
function of getting around an information space such as 
Internet, and performing information processing such as 
search for information, filtering, scheduling and the like 
on behalf of humans (software agent function). The in- 20 
teracttve agents have conversation with humans as if 
they are humans. Thus, they may be sometimes called 
anthropomorphic agents. 

[0028] The interactive apparatuses may have at least 
one of an agent function and a traveling function. 25 
[0029] Figure 2 shows an exemplary internal structure 
of the robot 1. 

[0030] An image recognition section 20 captures im- 
age from a camera 1 0 (image input section), recognizes 
the captured image, and outputs the recognized result 30 
to a processing section 50. 

[0031] A voice recognition section 40 captures voice 
from a microphone 30 (voice input section), recognizes 
the captured voice, and outputs the recognized result to 
the processing section 50. 35 
[0032] Figure 7 shows an exemplary internal structure 
of the voice recognition section 40. 
[0033] The voice input section 30 (microphone) con- 
verts voice into a voice signal waveform. The voice sig- 
nal waveform is output to the voice recognition section *o 
40. The voice recognition section 40 includes voice de- 
tection means 71 , comparison operation means 72, rec- 
ognition means 73, and a registered voice database 74. 
[0034] The voice detection means 71 cuts a part of 
the voice signaJ waveform input from the voice input sec- 45 
tion 30, which satisfies a certain standard, as a voice 
interval actually produced by a user, and outputs the au- 
dio signal waveform in the interval to the comparison 
operation means 72 as a voice waveform. Herein, a cer- 
tain standard for cutting out the voice interval may be, so 
for example, that power of the signal waveform in a fre- 
quency band of 1 kHz or less, which is generally a voice 
band of humans, is at a certain level or hither. 
[0035] In the registered voice database 74, voice 
waveforms of words which are expected to be uttered » 
by the user and the words are registered with the corre- 
spondences therebetween. 

[0036] The comparison operation means 72 sequen- 



tially compares voice waveforms input front the voice 
detection means 71 with the voice waveforms registered 
in the registered voice database 74. The comparison op- 
eration means 72 calculates the degree of similarity for 
each of the voice waveforms registered in the registered 
voice database 74, and outputs the calculated results to 
the recognition means 73. Herein, a method for compar- 
ing two voice waveforms may be a method of comparing 
totals of differences in power components at respective 
frequencies after the voice waveform is subjected to fre- 
quency analysis such as Fourier transform or the like, 
or may be a method in which DP matching is performed 
with an expand and contract in time being taken into ac- 
count in cepstrum feature quantity orMel cepstrum fea- 
ture quantity which is further subjected to polar coordi- 
nate transformation after the frequency analysis. More- 
over, for efficient comparison operation , the voice wave- 
forms registered in the registered voice database 74 
may be comparison factors used in the comparison op- 
eration means 72 (for example, power components of 
the respective frequencies). Further, among the voice 
waveforms registered in the registered voice database 
74, voice waveforms of voice produced unintentionally 
by the user, for example, cough, groan, and the like are 
registered, and, as the corresponding words, "uninten- 
tional voice production" is registered. Thus, it becomes 
possible to distinguish between the voice production in- 
tended by the user and the voice production which is not 
intended. 

[0037] The recognition means 73 detects the voice 
waveform which has the highest degree of similarity 
from the degrees of similarities of the respective voice 
waveforms input from the comparison operation means 
72. The recognition means 73 decides the word corre- 
sponding to the voice waveform detected from the reg- 
istered voice database 74 to convert the voice waveform 
into text, and output the text to the processing section 
50. When there is no significant difference among the 
similarities, it may determine that the input voice is noise 
and does not perform conversion from the voice wave- 
form into the text Alternatively, it may convert the voice 
waveform into the text such as "noise". 
[0038] Rgure 8a shows an exemplary internal struc- 
ture of the processing section 50 shown in Figure 2. 
[0039] The processing section 50 (processing means 
50a) searches a conversation database 140 based on 
the voice recognition results by the voice recognition 
section 40, and generates a responding sentence. The 
responding sentence is output to a speech synthesis 
section 100. The speech synthesis section 100 synthe- 
sizes the responding sentence into a speech. The syn- 
thesized speech is output from the audio output section 
110 such as a speaker. 

[0040] In the conversation database 140, patterns of 
conversation and rules for generating responding sen- 
tences. The conversation database 140 further stores 
the relationships between the words (keywords) uttered 
by the user and hearth conditions of the user. 
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[0041] Figure 3 shows exemplary relationships be- 
tween the keywords uttered by the user, which are 
stored in the conversation database 140, and the health 
conditions of the user. 

[0042] I n the example shown in Figure 3, the relation- 
ships between the keywords uttered by the user and the 
health conditions of the user are represented in a format 
of a table. For example, a row in this table indicates that 
keywords such as "sleepy", "tired", and "not feel like eat- 
ing" correspond to the health condition (physical condi- 
tion) of the user, "fatigue". A row 32 of the table shows 
that keywords such as "yes!" and "great!" correspond to 
the health condition (feeling) of the user, "pleasure". 
[0043] The way to represent the relationships be- 
tween the keywords uttered by the user and the health 
conditions of the user is not limited to that shown in Fig- 
ure 3. The relationships between the keywords uttered 
by the user and the health conditions of the user may 
be represented in any way. 

[0044] The processing section 50 (detection means 
50b) extracts a keyword from the voice recognition re- 
sult by the voice recognition section 40, and searches 
the conversation database 140 using the keyword. Con- 
sequently, the processing section 50 (detection means 
50b) detects the health condition of the user from the 
keyword. For example, when the keyword extracted 
from the voice recognition result is one of "sleepy", 
"tired", and "not feel like eating", the processing section 
50 (detection means 50b) determines that the health 
condition of the user is "fatigue" with reference to the 
table as shown in Figure 3. 

[0045] I nstead of or in addition to the above-described 
method using keywords, the health condition may be de- 
tected by detecting the level of the strength or deepness 
of the voice of the user based on the voice recognition 
result. For example, when the processing section 50 
(detection means 50b) detects that the level of the 
strength or deepness of the voice of the user equals to 
or lower than the predetermined level, the processing 
section 50 (detection means 50b) determines that the 
health condition of the user is "fatigue". 
[0046] Further, in addition to the voice recognition re- 
sult by the voice recognition section 40, the health con- 
dition of the user may be detected using the image rec- 
ognition result by the image recognition section 20. Al- 
ternatively, the health condition of the user may be de- 
tected by using only the image recognition result by the 
image recognition section 20. For example, when the 
processing section 50 (detection means 50b) detected 
that the user frequently blinks (or the user yawns), the 
processing section 50 (detection means 50b) deter- 
mines that the health condition of the user is "fatigue". 
[0047] As such, the processing section 50 (detection 
means 50b) may functions as detection means for de- 
tecting the health condition of the user based on the ut- 
terance of the user or the image recognition result. 
[0048] An information database 160 stores informa- 
tion such as today* s weather and news, knowledge such 



as various common knowledge, information regarding 
the user (owner) of the robot 1 (for example, information 
such as sex, age, name, occupation, character, hobby, 
date of birth, and the like), information regarding the ro- 
5 bot 1 (for example, information such as model number, 
internal structures and the like). The information such 
as today's weather and news is obtained by, for exam- 
ple, the robot 1 from outside via the sending/reoeiving 
section 60 (communication section) and the processing 
10 section 50, and stored in the information database 160. 
Further, the information database 160 stores the rela- 
tionships between the health conditions of the user and 
action patterns as an action pattern table 161. 
[0049] Figure 4 shows an exemplary action pattern ta- 
rs bie 1 61 stored in the i nf ormation database 1 60. The ac- 
tion pattern table 161 defines the relationships between 
the hearth condition of the user and the robot 1. 
[0050] In the example shown in Figure 4, the hearth 
condition of the user and the action pattern of the robot 
20 1 are represented in the format of a table. For example, 
a row 41 shows that the health condition of the user, 
"fatigue" corresponds to three kinds of action patterns 
of the robot 1 . Three kinds of action patterns are as fol- 
lows. 

25 

1) Selecting and reproducing contents: Select con- 
tents (software) which produce a "healing" or "hyp- 
notic" effect, and reproduce the selected contents 
(software) with a reproducing device 

30 

2) Preparing a bath: Prepare a bath in order to sug- 
gest the user to take a bath 

3) Selecting and preparing a recipe of food or drink: 
35 Select a recipe of food or drink which "increases the- 

appetite", and/or which is "nourishing", and prepare 
the food or drink following the selected recipe 

[0051 ] A row 42 in the table shows that the health con- 
40 dition of the user, "pleasure", correspond to the action 
pattern of the robot 1 , "gesture of 'banzai 1 (raising arms 
for cheering)". 

[0052] The way to represent the relationships be- 
tween the health conditions of the user and the action 

^5 patterns of the robot 1 is not limited to that shown in 
Figure 4. The relationships between the hearth condi- 
tions of the user and the action patterns of the robot 1 
may be represented in any way. 
[0053] Examples of the action patterns of the robot 1 

so include: selecting the contents (software) which match- 
es the hearth condition of the user and reproducing the 
selected contents (software) with a reproducing device; 
selecting a recipe of food or drink which matches the 
health condition of the user and preparing the food or 

55 drink following the selected recipe; preparing a bath; 
and telling joke for getting a laugh. 
[0054] The processing section 50 (action pattern de- 
ciding means 50c) searches the information database 
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160 (action pattern table 161) using the health condition 
of the user detected by searching the conversation da- 
tabase 140 in response to a timing signal t1 output from 
the detection means 50b. Consequently, the processing 
section 50 (the action pattern deciding means 50c) de- 
termines the action pattern of the robot 1 in accordance 
with the health condition of the user. For example, when 
the health condition of the user is "fatigue", the process- 
ing section 50 (action pattern deciding means 50c) de- 
termines one of the three action patterns defined in cor- 
respondence with "fatigue" as the action pattern of the 
robot 1 with reference to the table shown in Figure 4 
(action pattern table 161). 

[0055] Herein, the processing section 50 (action pat- 
tern deciding means 50c) can decide one of three action 
patterns as the action pattern of the robot 1 in various 
manner For example, when priorities may be allocated 
to three action patterns, the action pattern of the robot 
1 may be decided in descending order of priorities. The 
priorities may be varied depending on the time of the 
day. For example, the priority of "preparing a bath" may 
be made to be the highest during the time from 1 8:00 to 
22:00, the priority of "selecting and preparing a recipe 
of food or drink" may be made to be the highest during 
6:00 to 8:00, 11 :00 to 13:00, and 17:00 to 19:00, and in 
other time, the priority of "selecting and reproducing 
contents" may be made to be the highest. 
[0056] As described above, the processing section 50 
(action pattern deciding means 50c) functions as decid- 
ing means for deciding on the action pattern in accord- 
ance with the health condition of the user detected by 
the detection means 50b. 

[0057] The processing section 50 (execution instruct- 
ing means 50g) generates a control signal according to 
the decided action pattern in response to a timing signal 
t2 output from the action pattern deciding means 50c, 
and outputs the control signal to an operation control 
section 120. 

[0058] The operation control section 120 drives vari- 
ous actuators 130 in accordance with a control signal 
output from the processing section 50 (execution in- 
structing means 50g). Thus, it becomes possible to op- 
erate the robot 1 in a desired manner. 
[0059] For example, when the decided action pattern 
is the "gesture of 'banzai*", the operation control section 
1 20 drives an actuator (a part of the actuator 1 30) which 
moves "arms" of the robot 1 up and down in accordance 
with the control signal output from the processing sec- 
tion 50 (execution instructing means 50g). When the de- 
cided action pattern is "selecting and reproducing con- 
tents", the operation control section 120 may drive an 
actuator (a part of the actuator 130) for controlling "fin- 
gers of hands" of the robot 1 so as to hold a disc and 
set the held disc in a reproducing device in accordance 
with the control signal output from the processing sec- 
tion 50 (execution instructing means 50g). For example, 
a plurality of discs are arranged and stored in a rack in 
a predetermined order. 



[0060] As described above, the processing section 50 
(execution instructing means 50g) functions as execu- 
tion instructing means for instructing execution of the 
action pattern decided by the action pattern deciding 
5 means 50c to the operation control section 120. 

[0061] Alternatively, when the decided action pattern 
is "preparing a bath", the processing section 50 (execu- 
tion instructing means 50g) may control a remotecontrol 
section 70 so as to send a remote control signal to a hot- 

io water supply device. The hot-water supply device sup- 
plies an appropriate amount of hot-water of a desired 
temperature (or, supply an appropriate amount of water 
to a bath tab and then heat the water to the desired tem- 
perature) in accordance with a remote control signal. In 

'5 this case, the processing section 50 (execution instruct- 
ing means 50g) functions as instruction indicating 
means for indicating the execution of the action pattern 
decided by the action pattern deciding means 50c to the 
remote control section 70. 

20 [0062] Alternatively, when the decided action pattern 
is "selecting and reproducing contents", the processing 
section 50 (execution instructing means 50g) may con- 
trol a remote control section 70 so as to send a remote 
control signal to a reproducing device. The reproducing 

25 device selects the contents from discs set in the repro- 
ducing in accordance with the remote control signal for 
reproduction. If the reproducing device is connected to 
a disc changer which allows for a plurality of discs to be 
set, the reproducing device may select the contents 

30 from the plurality of discs in accordance with a remote 
control signal for reproduction. A list for selecting a mu- 
sical piece including all the musical pieces in a plurality 
of discs may be stored in a memory in the processing 
section 50. Alternatively, the reproducing device may 

35 read a list for selecting a musical piece of a disc from a 
header portion of the disc, and then store in a memory 
in the processing section 50 via the sending and receiv- 
ing section 60. In such a case, the processing section 
50 (execution instructing means 50g) functions as exe- 

40 cution instructing means for instructing execution of the 
action pattern decided by the action pattern deciding 
means 50c to the remote control section 70. 
[0063] Rgure 8b shows another exemplary internal 
structure of the processing section 50 shown in Rgure 

45 2. In the example shown in Rgure 8b, the processing 
section 50 (offering means Sue) makes an offer of the 
decided action pattern to the user by a speech before it 
instructs execution of the action pattern. For example, 
when the decided action pattern is "preparing a bath", 

so in response to the timing signal t2 output from the action 
pattern deciding means 50c, the processing section 50 
(offering means 50e) may generate interrogative sen- 
tence (offering sentence) such as "You look tired. Shall 
I prepare a bath for you?" with reference to the conver- 
ts sation database 140, and outputto the speech synthesis 
section 100. The speech synthesis section 100 synthe- 
sizes the interrogative sentence into a speech. The syn- 
thesized speech is output from the audio output section 
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110. 

[0064] Next, how the offering means 50e create offer- 
ing sentences will be described with reference to Figure 
9. The offering means 50e includes an offering sentence 
synthesis section therein. The conversation database 
140 includes an offering sentence format database 
therein. In the offering sentence format database, a plu- 
rality of offering sentence formats corresponding to a 
plurality of offer expressions are recorded and stored. 
Herein, "offer expressions" are words and expressions 
which indicate a cause (A) which motivates the offer and 
a response (B) to the cause, such as, "You're A, arent 
you? Shall I B7" or "You look A. Can I B?" as shown in 
Figure 9, for example. 

[0065] First, the offering means (offer synthesis sec- 
tion) 50e selects an offering sentence format which 
matches the "detected health condition" from the offer- 
ing sentence format database based on the "detected 
health condition" input from the detection means 50b 
and the "decided action pattern" input from the action 
pattern deciding means 50c. Next, the offering means 
(offer synthesis section) 50e synthesizes an offering 
sentence by inserting the "detected health condition" in- 
to A in the offering sentence format, and the "decided 
action pattern" into B. For example, when the "detected 
health condition" is "fatigue", and the "decided action 
pattern" is "preparing a bath", the offering means (offer 
synthesis section) 50e synthesizes an offering sen- 
tence, "You look tired. Shall I prepare a bath for you?". 
The offering sentence is output to the speech synthesis 
section 100. The speech synthesis section 100 synthe- 
sizes the offering sentence into a speech. The synthe- 
sized speech is output from the audio output section 
110. 

[0066] As described above, the processing section 50 
(offering means 50e) functions as offering means for 
making an offer of an action pattern decided by the ac- 
tion pattern deciding means 50c to the user by a speech 
before it instructs the execution of the action pattern by 
using the conversation database (offering sentence for- 
mat database) 140, the speech synthesis section 100, 
and the audio output section 110. 
[0067] The user gives an answer to the offer from the 
robot 1 whether to accept the offer or not. For example, 
the user gives an answer such as "yes", "yeah", "please 
do that" and the like as an indication to accept the offer 
(Yes). Alternatively, the user gives an answer such as 
"no", "no, thanks", "dont need that* and the like as an 
indication not to accept the offer (No). Such patterns of 
answers are previously stored in the conversation data- 
base 140. 

[0068] The processing section 50 (offer acceptance 
determination means 501) determines whether the an- 
swer of the user is an answer to accept the offer (Yes) 
or an answer not accept the offer (No) by analyzing the 
voice recognition result by the voice recognition section 
40 with reference to the conversation database 140 in 
response to a timing signal t5 output from the offering 



means 50e. 

[0069] As described above, the processing section 50 
(offer acceptance determination means 50f) functions 
as offer acceptance determination means for determin- 
5 ing whether the answer of the user is an answer to ac- 
cept the offer (Yes) or an answer not accept the offer 
(No) by using the voice recognition section 40 and the 
conversation database 140. 

[0070] Figure 8c shows another exemplary internal 

10 structure of the processing section 50 shown in Figure 
2. Whether it is necessary to make the offer of the de- 
cided action pattern to the user before execution of the 
action pattern may be determined. For example, by pre- 
viously setting an action offer necessity table 1 62 shown 

f 5 in Figure 1 1 where flags indicating necessities of offers 
are previously allocated to the action patterns in the ta- 
ble shown in Figure 4, the processing section 50 (offer 
necessity determination means 50d) can determine 
whether the offer is necessary or not in accordance with 

20 values of the flags. For example, the processing section 
50 (offer necessity determination means 50d) makes an 
offer of an action pattern to the user when the value of 
the flag allocated to the action pattern is "1" before it 
instructs execution of the action pattern, and does not 

25 make an offer of an action pattern to the user when the 
value of the flag allocated to the action pattern is "0* 
before it instructs execution of the action pattern. 
[0071] For example, regarding the action pattern of 
"preparing a bath", it is preferable that the offer to the 

30 user beforehand is required. Whether or not the user 
wants to take a bath or not largely depends on the mood 
at the time of the user. Thus, if the offer to the user be- 
forehand is not required, it may be intrusive. For exam- 
ple, regarding the action pattern of the "gesture of 'ban- 
as zaf ", it is preferable that the offerto the user beforehand 
is not required. If the user is asked for permission every 
time the banzai gesture is performed, it may look foolish. 
[0072] As described above, the processing section 50 
(offer necessity determination means 50d) functions as 

40 offer necessity determination means for determining 
whether or not it is necessary to make an offer of the 
decided action pattern to the user before it instructs ex- 
ecution of the action pattern by using the information 
database 160 (action offer necessity table 162). 

43 [0073] If the time of the day the action pattern is per- 
formed is always the same, or the action pattern is fre- 
quently performed, it is not desirable to make the offer 
of the action pattern to the user every time. On the other 
hand, regarding an action pattern which is not per- 

so formed usually, it is preferable to confirm whether the 
user wants execution of the action pattern by making an 
offer of the action pattern to the user before execution 
of the action pattern is instructed. 
[0074] With reference to Figure 1 0, the offer necessity 

55 determination means 50d which implements the above- 
described function will be described. A time distribution 
record storage section 90 includes a clock time meas- 
urement section 91, an integrating section 92, and a 
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time distribution database 93. The offer necessity deter- 
mination means 50d includes comparison deciding sec- 
tion therein. The clock time measurement section 91 re- 
ceives an input of the execution instructing means 50g, 
measures the clock time when the action pattern is per- 
formed, and outputs to the integrating section 92. The 
time distribution database 93 records and stores the 
number of times each of the action patterns is performed 
at every clock time. The integrating section 92 adds 1 
to the number of times recorded in the time distribution 
database 93 at the measured clock time every time it 
receives input from the clock time measurement section 
91 . The time distribution record storage section 90 ac- 
cumulates history information of action patterns per- 
formed at every clock time as such. The offer necessity 
determination means (comparison deciding means) 
50d has pre-set values, and, when it receives an input 
from the action pattern deciding means 50c, refers the 
number of times the action pattern is performed in the 
past at the clock time (or, in the time period) to the time 
distribution record storage section 90, and compares 
with the pre-set value. The comparison deciding section 
determines that it is necessary to make offer of the ac- 
tion pattern when the number of times the action pattern 
is performed in the past is smaller than the pre-set value, 
and determines that it is not necessary to make an offer 
of the action pattern when the number of times the action 
pattern is performed in the past is larger than the pre- 
set value. The determined results is output from the offer 
necessity determination means 50d as determination 
results of the offer necessity determination means 50d. 
[0075] As described above, the offer necessity deter- 
mination means 50d determines the necessity of mak- 
ing offer based on time distribution of the number of 
times the action pattern is performed. 
[0076] Figure 5 shows a procedure of process where 
the robot 1 detects the health condition of the user and 
instructs execution of an action pattern which matches 
the health condition of the user. 
[0077] Step ST1 : The health condition of the user is 
detected. 

[0078] For example, the processing section 50 (de- 
tection means 50b) extracts a keyword from the voice 
recognition result by the voice recognition section 40, 
and searches the conversation database 140 using the 
keyword. As a result, the processing section 50 (detec- 
tion means 50b) can detect the health condition of the 
user from the keyword. 

[0079] Hereinafter, an example of the conversation 
between the user and the robot 1 is shown. Herein, U 
denotes the utterance by the user, and S denotes to the 
speech of the robot 1. 

U: I'm tired today. 
S: Looks like that 

[0080] As in this example, when the user utters key- 
words such as "sleepy", "tired", and "not feel like eating", 



the processing section 50 (detection moans 50b) deter- 
mines that the health condition of the user is "fatigue". 
[0081] Step ST2: An action pattern is decided in ac- 
cordance with the health condition of the user detected 

5 in step ST1 . 

[0082] Forexample.theprocessingsectionSO (action 
pattern deciding means 50c) searches the information 
database 1 60 (action pattern table 1 61 ) using the health 
condition of the user. As a result, the processing section 

10 50 (action pattern deciding means 50c) can decide the 
action pattern corresponding to the health condition of 
the user. It is preferable that the action pattern is previ- 
ously set as estimating the demand of the user 
[0083] Step ST3: Whether it is necessary to make an 

'5 offer of the action pattern to the user before the instruc- 
tion of execution of the action pattern decided in step 
ST2 is determined by the offer necessity determination 
means 50d. 

[0084] When the determined result in step ST3 is 
20 "Yes", the process goes to step ST4, and, when the de- 
termined result in step ST3 is "No", the process goes to 
step ST6. 

[0085] Step ST4: The offer of the action pattern de- 
cided in step ST2 is given to the user by the offering 
2s means 50e before the execution of the action pattern is 
instructed. 

[0086] Hereinafter, an example of the conversation 
between the user and the robot 1 is shown. Herein, U 
denotes the utterance by the user, and S denotes to the 
30 speech of the robot 1 . 

S: You look tired. Shall I reproduce contents (soft- 
ware) having a healing effect? 
U: Yeah. 

35 

[0087] Step ST5: Whether or not the user give an an- 
swer to accept the action pattern offered by the robot 1 
in step ST4 is determined by the offer acceptance de- 
termination means 50f. 

[0088] When the determined result in step STB is 
"Yes", the process goes to step ST6, and, when the de- 
termined result in step ST5 is "No", the process goes to 
stepST7. 

[0089] Step ST6: Execution of the action pattern de- 
cided in step ST2 is instructed by the execution instruct- 
ing means 50g. 

[0090] Step ST7: The offered action pattern and the 
fact that the user did not accept (rejected) the offer are 
stored in the information database 160 as history infor- 
50 mation. 

[0091] The history information is referred to from the 
next time to decide on contents of an action pattern in 
step ST2 from the next time. The priority allocated to the 
action pattern which is not accepted by the user can be 
55 made lower. 

[0092] Instead of or in addition to step ST7, in the case 
where the offer is accepted by the user in step ST5, the 
offered action pattern and the fact that the user took up 
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(accepted) the offer may be stored in the information da- 
tabase 160 as history information. The history informa- 
tion is referred to from the next time to decide on con- 
tents of an action pattern in step ST2. The priority allo- 
cated to the action pattern which is accepted by the user 
can be made higher 

[0093] As described above, it is preferable to vary the 
priorities allocated to action patterns in accordance with 
whether the offered action patterns are accepted by the 
user or not. This allows reflecting habits and the like of 
the user in deciding on the action patterns. As a result, 
it becomes possible to improve the percentage that the 
action pattern decided by the robot 1 actually matches 
the health condition of the user. 
[0094] The user may make a counteroffer when the 
user did not accept the offer in step ST5. In such a case, 
the robot 1 receives the counteroffer and determines 
whether the counter offer is executable or not. When it 
is determined that the counteroffer is executable, the ro- 
bot 1 updates the relationshp between the health con- 
dition of the user and the action pattern of the robot 1 
stored in the information database 160 (for example, up- 
dates the priorities of the action patterns in the table 
shown in Figure 4, or, adds new patterns in the table 
shown in Figure 4), and then instructs execution of the 
counteroffer. When it is determined that the counteroffer 
is not executable, the robot 1 notifies of the user that 
"the counteroffer cannot be performed". In this way, by 
providing the counteroffer from the user, habits and the 
like of the user can be reflected in deciding on the action 
patterns. As a result, it becomes possfcte to improve the 
percentage that the action pattern decided by the robot 
1 actually matches the health condition of the user. 
[0095] In Figure 5, step ST3 may be omitted. In such 
a case, all the action patterns decided in accordance 
with the health conditions of the user are offered to the 
user before execution of the action patterns is instruct- 
ed. 

[0096] Further, in Figure 5, steps ST3, ST4, ST5, and 
ST7 may be omitted. In such a case, all the action pat- 
terns decided in accordance with the health condition of 
the user are instructed to be performed immediately 
without wafting for an answer from the user. 
[0097] As described above, according to the present 
embodiment, the health condition of the user is detect- 
ed, and the action pattern in accordance with the health 
condition of the user is decided. Thus, the user can be 
relieved from a burden of wearing various sensors. Fur- 
thermore, the user feels that the robot is an entity that 
cares about the health condition of the user (good 
friend). 

[0098] Further, a system to make an offer of the action 
pattern to the user before indicating execution of the ac- 
tion pattern may be employed. In such a case, the user 
has a final decision on whether to accept the offer or 
not. Thus, the user is not force by the robot to accept 
the offer, and has a high degree of freedom in judgment 
This allows suppressing runaway of the robot, and also 



for the user to feel familiar to the robot as a user-friendly 
entity. 

[0099] According to a survey conducted by JMA Re- 
search Institute Inc., the most poplar dream robot imag- 
5 ined by consumers was a "robot pet more like a real pet". 
Robots of coexistent or entertainment type closely re- 
lated humans' lives which share a living space with hu- 
mans are expected. 

[01 00] It could be understood that the robot as an ex- 

10 ample of the interactive apparatus according to the 
present invention is a friendly and useful robot closely 
related to humans' lives. Such a robot can help the life 
of the user and may be a good friend of the user. 
[0101] The contents (software) to be reproduced by 

'5 the reproducing device may include at least one of video 
data, audio data, and lighting control data. It is possible 
to reproduce audio data recorded on a recording medi- 
um (such as DVD) in synchronization with reproduction 
of video data recorded in the recording medium. It is also 

20 possible to reproduce lighting control data recorded on 
a recording medium (such as DVD) in synchronization 
with reproduction of audio data and/or video data. Such 
a synchronized reproduction allows to realize contents 
(software) having a significant "healing" effect and/or 

25 "hypnotic" effect. 

[0102] Figure 6 shows an exemplary structure of a re- 
producing apparatus 2100 which allows synchronized 
reproduction of the audio data and/or video data, and 
the lighting control data. The reproducing apparatus 

30 2100 is connected to an audio outputting device (for ex- 
ample, a speaker) and a video outputting device (for ex- 
ample, a TV). Thus , the reproducing apparatus 2100 
can change a lighting pattern of a lighting apparatus (for 
example, at least one of light intensity and color of light 

35 of the lighting apparatus) in conjunction with music and/ 
or video provided by a recording medium. 
[0103] The reproducing apparatus 2100 includes a 
controller 2220, an interface controller (l/F controller) 
2230, and a reading out section 2120. 

40 [0104] The controller 2220 controls the entire opera- 
tion of the reproducing apparatus 2100 based on an op- 
eration command from the user which is to be input into 
the l/F controller 2230 or a control signal provided from 
a decoding section 2140. 

45 [0105] The l/F control!er2230 detects an operation by 
the user (for example, a remote control signal from the 
remote control section 70( Figure 2)), and outputs an op- 
eration command corresponding to the operation (for 
example, a reproduction command) to the controller 

so 2220. 

[01 06] The reading out section 21 20 reads out infor- 
mation recorded on a recording medium 2110. 
[0107] The recording medium 2110 is, typically, a 
DVD (Digital Versatile Disk). However, the recording 
55 medium 21 1 0 is not limited to DVD. The recording me- 
dium 21 1 0 may be any type of recording medium. In the 
following description, an example in which the recording 
medium 2110 is a DVD will be described. In this case, 
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the reading out section 2120 is, for example, an optical 
pickup. 

[01 08] As a format for the data recorded i n the record- 
ing medium 21 10, a modified version of a format in con- 
formity with DVD-Image standard is used. Specifically, 5 
a format with a lighting pack (L_PCK) newly provided in 
VOBU is used. Data of L_PCK is data for outputting 
lighting control data in synchronization with presentation 
data. 

[0109] MPEG-2 (Moving Picture Experts Group 2) de- 10 
fines two types of schemes as a scheme for multiplexing 
any number of encoded streams and reproducing the 
streams in synchronization in order to be compatible 
with a wide range of applications. The two types of 
schemes are a program stream (PS) scheme and a *5 
transport stream (TS) scheme. Digital storage media 
such as DVD employs the program stream (PS) 
scheme. In the following description, the program 
stream (PS) scheme defined by MPEG-2 is abbreviated 
as "MPEG-PS scheme", and the transport stream (TS) 20 
scheme defined by MPEG-2 is abbreviated as 
"MPEG-TS scheme". 

[0110] Each of NV.PCK, A_PCK, V_PCK, and 
SP_PCK employs a format in conformity with the 
M PEG-PS scheme. Thus, L_PCK also employs a format 25 
in conformity with the MPEG-PS scheme. 
[0111] The reproducing apparatus 2100 further in- 
cludes a stream data generation section 2130, and the 
decoding section 21 40. 

[0112] The stream data generation section 21 30 gen- so 
erates stream data including encoded AV data and en- 
coded lighting control data based on the output from the 
reading out section 2120. Herein, "encoded AV data* re- 
fers to data including at (east one of encoded audio data 
and encoded video data. 35 
[0113] The stream data generated by the stream data 
generation section 2130 has a format in conformity with 
the MPEG-PS scheme. Such a stream data can be ob- 
tained by, for example, receiving information recorded 
in the DVD 2120 in the form of an RF signal, digitizing *o 
and amplifying the RF signal, and performing EFM and 
demodulation process. The structure of the stream data 
generation section 2130 may be same as the one 
known. Thus, detailed description is omitted here. 
[0114] The decoding section 2140 includes a decern- 45 
position section 2150, an AV data decoding section 
2160, a lighting control data decoding section 2170, an 
STC generation section 2180, and a synchronization 
controller (control section) 2190. 

[0115] The decomposition section 2150 receives so 
stream data having a format in conformity with the 
MPEG-PS scheme from the stream data generation 
section 21 30, and decomposes the stream data into en- 
coded AV data and encoded lighting control data. Such 
decomposition is performed with reference to an identi- 55 
fication code in a PES packet header (stream_id). The 
decomposition section 2150 is, for example, a demulti- 
plexer. 



[01 1 6] The AV data decoding section 21 60 outputs AV 
data by decoding the encoded AV data. Herein, "AV da- 
ta" refers to data including at least one of audio data and 
video data. 

[01 17] The AV data decoding section 21 60 includes: 
a video buffer 21 61 for temporarily storing encoded vid- 
eo data which is output from the decomposition section 
21 50; a video decoder 21 62 for outputting video data by 
decoding the encoded video data; an audio buffer 2163 
for temporarily storing encoded audio data which is out- 
put from the decomposition section 2150; and an audio 
decoder 2164 for outputting the audio data by decoding 
the encoded audio data. 

[0118] The lighting control data decoding section 
2170 outputs the lighting control data by decoding the 
encoded lighting control data. Herein, "lighting control 
data" is data for controlling a plurality of pixels included 
in the lighting apparatus. 

[0119] The lighting control data decoding section 
2170 includes: a lighting control buffer 2171 for tempo- 
rarily storing the encoded lighting data which is output 
from the decomposition section 2150; and a lighting de- 
coder 21 72 for outputting the lighting control data by de- 
coding the encoded lighting control data. 
[0120] The STC generation section 2180 generates 
STC (System Time Clock). STC is obtained by adjusting 
(increasing or decreasing) a frequency of a reference 
clock of 27MHz based on SCR. STC is a reference time 
used for encoding data which is reproduced when the 
encoded data is decoded. 

[0121] The synchronization controller 2190 controls 
the AV data decoding section 2160 and the lighting con- 
trol data decoding section 2170 such that the timing for 
the AV data decoding section 2160 to output AV data 
and the timing for the lighting control data decoding sec- 
tion 2170 to output the lighting control data are in syn- 
chronization. 

[0122] Controlling such a synchronized reproduction 
is achieved by, for example, controlling the video decod- 
er 2162 such that an access unit of video data is output 
from the video decoder 21 62 when STC and PTS match, 
controlling the audio decoder 2164 such that an access 
unit of video data is output from the audio decoder 21 64 
when STC and PTS match, and controlling the lighting 
decoder 2172 such that an access unit of video data is 
output from the lighting decoder 2172 when STC and 
PTS match. 

[0123] The synchronization controller 2190 may con- 
trol the AV data decoding section 2160 and the lighting 
control data decoding section 2170 such that the timing 
for the AV data decoding section 21 60 to decode AV da- 
ta and the timing for the lighting control data decoding 
section 2170 to decode the lighting control data are in 
synchronization. 

[0124] Controlling such a synchronized reproduction 
is achieved by, for example, controlling the video decod- 
er 21 62 such that an access unit of video data is decod- 
ed by the video decoder 2162 when STC and DTS 
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match, controlling the audio decoder 2164 such that an 
access unit of video data is decoded by the audio de- 
coder 2164 when STC and DTS match, and controlling 
the lighting decoder 2172 such that an access unit of 
video data is decoded by the lighting decoder 2172 5 
when STC and DTS match. 

[0125] As described above, in addition to controlling 
the timing to output access units of video data, audio 
data, and lighting control data, or, instead of controlling 
the timing to output access units of video data, audio' 10 
data, and lighting control data, controlling the timing to 
decode access units of video data, audio data, and light- 
ing control data may be performed. This is because, 
sometimes, the timing (order) to output the access units 
and the timing to decode the access unit are different 
from each other. Such a control enables synchronized 
reproduction of video data, audio data, and lighting con- 
trol data. 

[01 26] The video data output from the video decoder 
2162 is output to an external device (for example, TV) 20 
via an NTSC encoder 2200. The video decoder 2162 
and the TV may be directly connected to each other via 
an output terminal 2240 of the reproducing apparatus 
2100, or may be indirectly connected via a home LAN. 
[0127] The audio data output from the audio decoder 25 
2164 is output to an external device (for example, 
speaker) via a digital to analog converter (DAC) 2210. 
The audio decoder 21 64 and the speaker may be direct- 
ly connected via an output terminal of the reproducing 
apparatus 2100, or may be indirectly connected via a 30 
home LAN. 

[0128] The lighting control data output from the light- 
ing decoder 2172 is output to an external device (for ex- 
ample, lighting apparatus). The lighting decoder 2172 
and the lighting apparatus may be directly connected 3s 
via an output terminal 2260 of the reproducing appara- 
tus 2100, or may be indirectly connected via a home 
LAN. 

[01 29] The stream data generated by the stream data 
generation section 2130 may include encoded sub-vid- *o 
co data, or may include navigation data. For example, 
when the stream data include the encoded sub-video 
data and the navigation data, the decomposition section 
2150 decomposes the stream data into the encoded 
sub-video data and navigation data Although not shown 
in Figure 6, the decoding section 2140 may further in- 
cludes a navipaok circuit, a sub-picture decoder, and a 
closed caption data decoder. The navipack circuit gen- 
erates a control signal by processing the navigation da- 
ta, and outputs the control signal to the controller 2220. so 
The sub-picture decoder decodes the encoded sub-vid- 
eo data and outputs the sub-video data to the NTSC en- 
coder 2200. The closed caption data decoder decodes 
the encoded closed caption data included in the encod- 
ed video data and outputs the closed caption data to the ss 
NTSC encoder 2200. Since the functions of these cir- 
cuits are known and are not related to the subject matter 
of the present invention, the detailed description thereof 



is omitted. As described above, decoding section 2140 
may include a known structure which is not shown in 
Figure 6. 

[0130] As shown in the above description, according 
to the reproducing apparatus 2100 shown in Figure 6, 
a reproducing apparatus which allows that the lighting 
control data recorded on a recording medium is repro- 
duced in synchronization with reproduction of the audio 
data and/or video data recorded on the recording medi- 
um. By connecting the audio outputting device (for ex- 
ample, speaker), the video outputting device (for exam- 
ple, TV), and the lighting apparatus to the reproducing 
apparatus, it becomes possible to change lighting pat- 
tern in conjunction with music and/or video provided by 
the recording medium. Examples of the lighting patterns 
having a "healing" effect include a lighting pattern rep- 
resenting sunlight passing between tree branches. 

INDUSTRIAL APPLICABILITY 

[0131] As described above, according to interactive 
apparatus of the present invention, the health condition 
of the user is detected, and the action pattern in accord- 
ance with the health condition of the user is decided. 
Thus, the user can be relieved from a burden of wearing 
various sensors. Furthermore, the user feels that the in- 
teractive apparatus Is an entity that cares about the 
health condition of the user (good friend). As a result, 
the value of the interactive apparatus is increased, and 
satisfaction and a desire for possession of the user to- 
ward the interactive apparatus are increased. 



Claims 

1 . An interactive apparatus, comprising: 

detection means for detecting a health condi- 
tion of a user, 

deciding means for deciding on an action pat- 
tern in accordance with the health condition of 
the user detected by the detection means; 
execution instructing means for instructing ex- 
ecution of the action pattern decided by the de- 
ciding means; 

offering means for making an offer of the action 
pattern to the user with a speech before in- 
structing execution of the action pattern decid- 
ed by the deciding means; and 
determination means for determining whether 
an answer of the user to the offered action pat- 
tern is an answer to accept the offered action 
pattern or not, 

wherein the execution instructing means in- 
structs execution of the offered action pattern when 
the answer of the user is determined to be the an- 
swer to accept the offered action pattern. 
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2. An interactive apparatus according to claim 1, 
wherein the detection means detects the health 
condition of the user based on utterance of the user. 

3. An interactive apparatus according to claim 2, 
wherein the detection means detects the health 
condition of the user based on keywords uttered by 
the user. 

4. An interactive apparatus according to claim 1 , fur- 
ther comprising offer necessity determination 
means for determining whether it is required to 
make an offer of the action pattern to the user before 
instructing execution of the action pattern decided 
by the deciding means, 

wherein the offering means makes an offer of 
the action pattern to the user with a speech when it 
is determined that making an offer of the action pat- 
tern to the user is required before instructing exe- 
cution of the action pattern. 

5. An interactive apparatus according to claim 4, 
wherein the offer necessity determination means 
determines necessity of making an offer in accord- 
ance with a value of a flag indicating a necessity of 
making an offer which is previously allocated to the 
action pattern. 

6. An interactive apparatus according to claim 4, 
wherein the offer necessity determination means 
determines necessity of making an offer based on 
time distribution of the number of times the action 
pattern is performed. 

7. An interactive apparatus according to claim 1, 
wherein the deciding means decides one of a plu- 
rality of action patterns to which priorities are re- 
spectively allocated as an action pattern in accord- 
ance with the health condition of the user, and 
changes the priority allocated to the action pattern 
in accordance with whether or not the action pattern 
is accepted by the user. 

8. An interactive apparatus according to claim 1 , fur- 
ther comprising storage means for storing the ac- 
tion pattern in accordance with the health condition 
of the user, 

wherein the deciding means decides on the 
action pattern by using the action pattern stored in 
the storage means. 

9. An interactive apparatus according to claim 1, 
wherein the action pattern offered by the offering 
means to the user includes selecting contents to be 
reproduced by a reproducing device. 

10. An interactive apparatus according to claim 9, 
wherein the contents include audio data, video da- 



ta, and lighting control data, and the reproducing 
device changes at least one of light intensity and 
color of light of a lighting apparatus based on the 
lighting control data. 

5 

11. An interactive apparatus according to claim 1, 
wherein the interactive device has at least one of 
an agent function and a traveling function. 

*o 12. An interactive apparatus according to claim 1, 
wherein the health condition of the user represents 
at least one of feelings of the user and a physical 
condition of the user. 

'5 13. An interactive apparatus, comprising: 

. a voice input section for converting a voice pro- 
duced by the user into a voice signal, 
a voice recognition section for recognizing 

20 words uttered by the user based on the voice 

signal output from the voice input section; 
a conversation database in which words ex- 
pected to be uttered by the user are previously 
registered, and which stores correspondences 

25 between the registered words and the health 

condition of the user; 

detection means for detecting the health condi- 
tion of the user by checking the words recog- 
nized by the voice recognition section against 

30 the words registered in the conversation data- 

base, and deciding on the hearth condition of 
the user in accordance with the checking result; 
deciding means for deciding on an action pat- 
tern in accordance with the hearth condition of 

35 the user detected by the detection means 

based on an action pattern table storing corre- 
spondences between the health condition of 
the user and action patterns of the interactive 
apparatus; 

<o execution instructing means for instructing ex- 

ecution of the action pattern decided by the de- 
ciding means; 

offering means for synthesizing an offering sen- 
tence based on an output result of the detection 

45 means and an output result of the deciding 

means and making an offer of the action pattern 
to the user with a speech before instructing ex- 
ecution of the action pattern decided by the de- 
ciding means; and 

so determination means for determining whether 

an answer of the user to the offered action pat- 
tern is an answer to accept the offered action 
pattern or not, 

55 wherein the execution instructing means in- 

structs execution of the offered action pattern when 
the answer of the user is determined to be the an- 
swer to accept the offered action pattern. 
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14. An interactive apparatus according to claim 13, fur- 
ther comprising: 

means for receiving an action pattern which is 
counter-offered by the user with respect to the s 
offered action pattern; 

means for the interactive apparatus to deter- 
mine whether the counter-offered action pat- 
tern is executable or not; and 
means for updating the correspondences be- 10 
tween the health condition of the user and the 
action patterns of the interactive apparatus 
which are stored in the action pattern table 
when the interactive apparatus determines that 
the counter-offered action pattern is executa- is 
ble. 
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Sleepy, tired, not feel like eating 
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Normal 


Fine, hungry 


Good 
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